1. Field of the Invention
The present invention relates to determining eye positions and more particularly to a reliable, non-invasive system for monitoring an individual, processing images of the individual and providing feedback to the individual.
2. Description of the Prior Art
Monitoring of sleepiness and sleep has traditionally been of interest to psychologists and neurophysiologists studying behavioral effects of sleep deprivation, sleep disorders, etc. Recently, there has been a lot of interest in monitoring sleepiness and fatigue on the job--and the human error that results from it. Some categories of professionals that have directly stirred this interest are medical residents, truck and train drivers, airline pilots, air traffic controllers, assembly line workers involved in monotonous jobs, etc. The following examples and statistics are self explanatory: In 1978, a commercial airliner scheduled to land at Los Angeles International Airport passed over the airport at 32,000 feet and headed out over the pacific. Flying on autopilot, the plane was about 100 miles out to the sea before the air traffic controllers found a way to sound an alarm in the cockpit. The entire crew had fallen asleep on the flight deck. In the United States, an estimated total of 6,500 automobile related deaths every year--13 percent of the annual toll--are caused by drivers falling asleep at the wheel. Fatigue and human error remains the biggest cause of automobile accidents. While the safety of automobile drivers and passengers has been getting increased attention in recent years (air bags and anti-lock brakes are becoming available and affordable), car manufacturers have begun to look at fatigue sensing only recently. In a recent CNN survey of 1000 truck drivers, 3 out of 5 drivers admitted to have fallen asleep behind the wheel in the month preceding the survey. Truck drivers commonly exceed the maximum permitted 80 hours a week of driving.
While sociological implications of monitoring sleepiness and sleep on the job remain uncertain and need a serious consideration, technological progress is well on its way to make such monitoring possible. Frequently, an electroencephalogram (EEG) is used in conjunction with an electrooculogram (EOG) to measure eye movements and an electromyogram (EMG) to measure the muscle tonus in the chin. This is described in "Work Hours And Continuous Monitoring Of Sleep", Akerstedt, T., In Broughton, R. J., and Ogilvie, R. D., Sleep, Arousal and Performance, Birkhauser, Boston, 1991. Eye movements are particularly useful in detecting the accurate onset of sleep as described in "Eye Movements And The Detection of Sleep Onset", Ogilvie R. D., McDonagh, D. M. and Stone, S. N., Psychophysiology, Vol. 25, No. 1, pp. 81-91, 1988. Another alternative has been developed by Nissan which includes using distinctive steering patterns produced by a fatigued driver. This is discussed in "Electronic Applications For Enhancing Automotive Safety", Aono, S., Vehicle Electronics in the 90's: Proceedings of the International Congress on Transportation Electronics, pp. 179-186, Oct., 1990.
While technological advances have made EEG based monitoring of sleep quite convenient (portable EEG-sleep recorders of pocket size are available), it is essentially an invasive technique. The subject has electrodes attached to his/her head. In applications such as monitoring sleepy behavior of a car driver, a non invasive technique is more desirable. Visual monitoring of the eyes appears to have the promise of providing such a non invasive technique. Specifically of interest is the monitoring of drivers (of cars, trucks, trains, etc.) and the ability to raise an alarm when the driver appears sleepy. For visual monitoring, one obvious and natural choice is to use the motion of a person's eye to detect sleepy behavior. This approach must be done correctly and reliably.
The correlation between the behavior of the eye and sleep dates back to very early days of sleep research. Miles, in his seminal paper of 1929 wrote: ". . . the contrast between alertness and drowsiness is nowhere more evident than in the condition and behavior of the eye . . . " Miles filmed rolling eye movements during drowsiness and used them to characterize sleep. While some of the earlier work on monitoring the eyes was done visually (by recording movies of the eye on a film), it was soon replaced by EEG and EOG based monitoring.
Practically none of the current research on sleep behavior uses visual monitoring of the eyes. This is partly because of high variability in the correlation between external appearance of the eye and physiological stages of sleep, as well as because of unavailability of equipment that can provide quantitative data. However, based on observation of sleep subjects as well as interaction with some of the experts, the following statements can be made regarding the behavior of the eyes during alertness, sleepiness and onset of sleep:
1. In a state of alertness, human eyes blink. The average blink rate (average number of blinks per minute) varies significantly across individuals. Also, for a given alert individual, the blink rate can vary as a function of time, depending on factors such as nervousness. The blink duration (the time it takes for the eyelid to close and open again) is of the order of one third of a second. PA1 2. The blink rate of the eye increases during sleepiness. In effort to fight sleep, the subjects often squint their eyes, and blink frequently. While the average blink rates of individuals differ significantly, there is a marked change of blink rate from alertness to sleepiness across the population. The blink rate can increase by a factor of two or more from alertness to sleepiness. PA1 3. During sleepiness, the eyelids become heavy and behave sluggishly. The time it takes for eyelids to close during sleepiness is much longer than the blink duration; it is of the order of a few seconds (in the range of one to four seconds). At the onset of sleep, the eyelid may or may not open after it closes. PA1 4. It is possible to be asleep with eyes open completely. A small fraction of the population exhibits this behavior. PA1 5. Slow Eye Movements (SEM) appear during sleepiness and disappear at the beginning of behaviorally and physiologically defined sleep. This pattern is a consistent indicator of sleep onset over large populations.
For many visual monitoring and surveillance applications, it is important to determine human eye positions from an image sequence containing a human face. Once the human eye positions are determined, all of the other important facial features, such as positions of the nose and mouth, can easily be determined. The basic facial geometric information, such as the distance between two eyes, nose and mouth size, etc., can further be extracted. This geometric information can then be used for a variety of tasks, such as for recognizing a face from a given face database. The eye localization system can also be directly used for detecting the sleepy behavior of a car driver.
Some techniques exist for eye localization based on the Hough transform, geometry and symmetry checks and deformable models. Most of these techniques are not sufficiently robust against shape changes. These systems also require an extensive amount of computer processing time. Furthermore, none of these existing systems can locate eyes when the eyes are closed.